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ABSTRACT 


Partial realizations of sequences over a finite field are 
obtained, using Berlekamp-Massey algorithm in both one dimension 
and two dimensions. This representation is applied to compress the 
image data. It has been established, that, through deterministic 
modelling of a stochastic behaviour, data compression cannot be 
achieved. However, by employing a fidelity criterion it was shown 
that some compression can be achieved. The introduction of error 
bound to two dimensional sequences could not be done because of 
some implementation problems. To get an optimal representation in 
the one dimensional case, the problem was shown to be NP-hard. 








€mci^ S^eacA^U 



ACKNOWLEDGEMENTS 


I am grateful to my thesis supervisor;, Dr. S.K. Mullick,*' for 
suggesting the topic for study. His constant encouragement, 
inspiration and constructive criticism helped a long way in 
shaping the thesis to the present form. 

I thank Motorola Inc. , for providing me financial 

assistance . 

The stimulating discussions with Madhu and Udaya are 
gratefully acknowledged. 

I wish to thank Ramprasad, for helping me in taking the hard 
copies of the images. 

I thank Ravi, Perraju and Ramudu, for helping me in the 
preparation of the thesis. 

There are any number of people who have been supportive 
throughout my stay at IITK. Particular mention goes to Hari, 
Babu, Srikanth, Murali, Reddy, Kalari, Das, Sandy, Arun. 



CONTENTS 


1 Introduction 1 

1.1 Introduction 1 

1.2 Modelling of a stochastic time 

ser i es 2 

1.3 Statement of the problem 7 

1.4 Overview of the thesis 8 

2 1-D partial realizations with an error 

criterion g 

2.1 Introduction 9 

2.2 Berlekamp-Massey algorithm 13 

2.3 Introduction of error criterion 17 

3 2-D partial realizations 24 

3. 1 Structure and properties of two 

dimensional systems 24 

3.2 Definitions and notations 29 

3. 3 2-D Berlekamp-Massey algorithm 32 

4 Applications of partial realizations to 

data conqjression 39 

4.1 Applications of 1-D partial 

realizations to image compression 39 

4.2 Application of 2-D partial 

realizations to compression 44 

4.3 Drawbacks of using partial 

realizations for data compression 44 

5 Conclusion SO 

References S3 


Appendi x 


56 



LIST OF FIGURES 


^ LFSR of length L that generates the given sequence 12 

2.2 Berlekamp-Massey algorithm 16 

2.3 Linear compplexity profile of a sequence 18 

2.4 Modified Berlekamp-Massey algorithm with 

er r or bound 23 

3. 1 Past and future of a point which do not divide 

the two dimensional plane into seperation sets. 25 

3. 2 Past and future of a point which divide the 

two dimensional plane into seperation sets. 27 

3. 3 Total ordering on 29 

3. 4 Subsets of Z 30 

o 

3. 5 A 2-D binary array 32 

3.6 The sets S and ACFD of a 2-D binary array CAD 34 

p 

3.7 A binary two dimensional array 37 

4.1 Compressed images using Bm algorithm with 

error criterion 41 

4.2 Compressed images using modified representation 

using partial realizations with error 
criterion 43 



4. 3 Compressed images using brute force method 



LIST OF TABLES 


3.1 Sample calculation of 2-D BM algorithm for 
the array shown in Fig. 3.7 


38 


4.1 Compressions obtained for various error values 
using BM algorithm with error bound 

4.2 Distribution of LFSR lengths using BM 
algorithm with error bound 

4.3 Compressions obtained for various error values 
using modified representation of sequences using 
partial realizations with error criterion 

4.4 Compressions obtained using brute force method 


40 

40 

42 

48 



CHAPTER 1 


INTRODUCTION 


1.1 Introduction 

Source coding as opposed to error control coding aims at 
removing the redundancy in the data . The present day source 
coding techniques may be classified into two broad catagories. One 
is source coding with error and the other one being distortionless 
coding. Linear predictive coding and transform coding come into 
the former category C13] . In these techniques for fixed 

compression ratios, the distortion is minimized by taking certain 
norm which is the mean square error. Run length coding and 
Ziv-Lempel coding come into the latter class Cl 1,24]. 

Source coding techniques have their theoretical basis in 
information theory. A mathematical treatment of the measure of 
information was given by Sliannon. His approach is what is 
presently known as the classical information theory. In 196S, yet 
another concept of information came into picture, based on the 
complexity theory. Chaitin and Kolmogorov [5,6,7] formalized the 
information in terms of complexity. Their approach has led to a 
now area in information theory , called the algorithmic approach 
to information theory. The study of this approach is important to 
us because it gives the relation between randomness and 


compress! bi 1 i ty . 
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1*2 Modelling of a stochastic time seriest 

In this section section two examples are presented which show 
that by using a deterministic model to represent a random series 
compression cannot be achieved. 

1.2.1 Chaitin*s Examples 

In this section the relation between randomness and 
compressibility is described. The following discussion is taken 
from the works of Chaitin, Kolmogorov C5,6»7]. 

Almost everyone has an intuitive notion of what random number 
is. For example consider the following two series of binary 
di gits. 

01 01 01 01 01 01 01 01 01 01 
01 1 001 1 001 1 00001 1 01 0 

The first one is obviously constructed according to a simple rule. 
Inspection of the second series of digits yields no comprehensive 
pattern. The arrangement seems haphazard. In otherwords it is a 
random assortment of ones and zeros. Consider a probabilistic 
experiment of tossing a fair coin 20 times. The two sequences 
given above occur with a probability of 2 . If the above 

sequence is an outcome of a random experiment, then both these 
sequences, would have to be considered as random and indeed so, 
all the other sequences. But this clearly goes against our 
intuition. 

Chaitin tried to define randomness in terms of the pattern it 
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follows. Thus a sequence is said "to be random if the length of the 
shortest description for the sequence is equal to the length of 
the sequence itself, i.&. , for a random sequence it is better to 

describe the entire sequence as it is, rather than searching for a 
pattern. Thus randomness and compressibility are inversly related. 

Consider a machine M, which takes sequences over a finite 
alphabet as input and outputs a program which contains a 
description of the sequence. The shorter the program is, the 
higher the compressibility of the sequence and vice versa. When a 
random sequence is given as input to this machine, the output 
program will have a length equal to the length of the sequence. 

Definition 1.1: A sequence is said to have a complexity j if 
the length of the shortest program required to describe the 
sequence i s equal to j . 

Theorem 1.1: Consider a machine which takes sequences of 
length n, over a finite alphabet of cardinality p , and outputs a 
sequence, C whose values were assumed to be taken from the same 
alphabet as the input sequence^ which represents the shortest 


description of the input sequence, 
are of complexity loss than j are 


The number of sequences whi ch 



p -1 
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complexity less than j is very small for very large length" of the 
sequence. This implies that for large values of r> Ijigher 
compression ratios cannot be achieved unless v/e have ^ a 
probabilistic description of the given data. 

1.2.2 Gaines’ model: 

Consider a second example. Suppose that we are observing 
a time series of O’s and I’s as the behaviour of certain dynamical 
system- the complexity of which we intend to compress , Suppose the 
modeler assumes a priori that the observed series is g-onerated by 
a causal system and therefore forms a deterministic model of 
it. For any finite length N of the observed time series, there will 
be an unbounded number of finite state machines whose input -output 
behaviour, starting from a specific state, is identical to that 
observed; out of these models there will be some tliat no 

other machine has a smaller number of states. An optim-"'! causal 
model is that which is a minimal -state machine model. 

Consider binary sequences of length N. There are 2 such 
sequences . Let us enumerate the different S— state deterministic 
automata that act as mod'll s for these sequences. The general form 
of the model will be a transient chain followed by a cycle. If 
there are S ' atec to be filled with two symbols O and 1, end the 
cycle ':an co. em in any state, there are atmost S2 models. Some 
of the ■? models r> y '‘">nerate the same sequence, making ati upper 
bound on the numb' ' of models, so 

= sa** > 2*" 


M 


. . . Cl . 2D 
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Now we will evaluate the mean number of states in the minimal 
form of these M models. 

Let MCRD be the probability of finding models with states 
from R=1 to R. The average value of MC RD for R between 1 and S-1 
is given by. 


mcr:> = ^ E R 

S-1 R=1 ACR3 


. . . Cl. 3D 


where 


AMCRD 


ACRD 


is the fraction of models with state R 


Therefore MCRD= E RfR-2*^ - CR-ID.S*^"^] 

s-1 R=1 


. . . C 1 . 4D 


L»t fj^ bQ thQ avoragQ numbar of the stata S. Since there are 

u B models like that » 

^ s 


/j 2® > H 2 ® > MCRD 


, . . Cl . 6D 


1 1 S“"l p p p 


. Cl . 7D 


Gaines’ simplification of the summation in the above 
inequality was wrong. His simplification gave the following 
inequality. 




. . .C1.8D 
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but we found that 




a 2 

“ Cs-132^"^ ^ 


. . . Cl . 8:) 


from the inequality C1.2Z) 

S > N - log^N-2 . . . Cl . 9D 

Ther ef or e , 

N - log^N - 2 
s-2 ^ 

/J > — > 2 ... C 1 . 1 05 




Expected number of states of models 
Number of observations 



> 


^ log^N+a 
2 “ N 


. . . C 1 . 1 1 5 


1 

Since Gaines ignored — in the summation, he concludes that 
Rj^->1 as N->oo . But the result we have got says that we can get 
compressions upto 50?^, by using this model. This is due to the 
fact that the RHS in the inequality is a lower bound and is 
heavily underestimated. 

The conclusion derived from the above t..o examples is that 
for large values of N it is difficult to form a causal model for th 
given series whose complexity is less than N. Infact we can say 
that, such a model doesn't exist, because the existence of such a 
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model clearly violates the fundamental theorem of information 
theory, which states that the entropy is a lower bound for the 
expected complexity of a code. 

To alleviate this problem, we have taken the following steps. 

Ci3 The sequence is broken into subsequences of smaller 
length. Each block of these sequences is separately coded. 

CiiD An error criterion was selected, so that the 
reconstructed sequence at the decoding end will be within some 
error limit of the original sequence. This gives an improvement in 
the compression ratios at the expense of losing some information 
which for the objective at hand can be tolerated by a machine or a 
human. This technique can be mainly used in the coding of image 
and speech signals, in the absence of the knowledge of the 
stati sti cs of the gi ven si gnal . 

We have considered partial realizations as an attractive 
choice for the aforesaid problem. Though they may not be optimal 
in Chai tin’s sense of shortest description, the algorithms are 
simple and hence the choice. 

1.3 Statement of the problenu 

The problem that is attempted in this thesis is to study the 
feasibility of some well known representations from linear system 
theory visf. rational approximations to data with a fidelity 
criterion. CXir aim is to fix the amount of distortion and to try to 
achieve higher compression ratios. Similar techniques have been 
employed in the probabilistic source coding techniques by placing 
an upper bound on the amount of distortion and minimising the 
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entropy [2]. Since our techniques are data dependent it is 
difficult to rank these with the existing techniques- 

1.4 Overview of the thesis: 

Chapter 2 describes the 1-D partial realizations, 
Berlekamp-Massey algorithm, the expected complexity of the 
rational approximation scheme, the introduction of error criterion 
for achieving data compression. 

Chapter 3 describes an introduction to 2-D systems, 2-D 
Partial realizations , Sakata’s extension of Berlekamp-Massey 
algorithm to two dimensions . 

Chapter 4 describes the applications of 1-D and 2-D 
partial realizations to data compression, the drawbacks of these 
representations and possible improvements. 

The conclusions are presented in chapter S. 



CHAPTER 2 


1-D PARTIAL REALIZATIONS WITH AN ERROR CRITERION 


2.1 Introduction: 


The objective of the present chapter is the study of the 
models which view the given data as the impulse response of a 
minimum order discrete linear filter with data and the 
coefficients confined to a finite field. This representation is 
referred to as minimal partial realization. 

The minimal partial realization is data specific and provides 
a transfer function PC 2D /QC 2D of a linear discrete filter with 
degree of PC 2D less than that of QC2D and C3C2D having a degree 
which is minimal over the family of all such realizations. The 
problem may not admit of a unique solution and many such 
realizations may bo p>ossible. Since with each rational function 
PC2D/C5CZ!) we may associate a formal power series by means of a 
long devision* e. g. 


PCZ> 

CJCZ> 


ss a 2 "^a 2 
1 2 


+a 2 

N 


.-K 


C2. ID 


<a^. a^, ,a^ represent the impulse response sequence> 

This implies that for a given sequence a^, a^ ,a^ 

PC ZD /QC 2D is a realization if and only if the first N coeffi cents 
of the formal power series of PCZD/QCZD are exactly the numbers 

a ,a ,a . In this sense a ,a^ ,a^, is viewed as a 

partial s&cipji 0 nc& or a subsequence of a certain fixed sequence and 
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PCZD/QC2D as a partial relization. In this sense the partial 
realization problem is the same as Pad6 representation problem. To 
obtain the minimal partial realization, we must some how find the 
right infinite extension sequence among many that may be possible. 
In this process we must not make any extrinsic assumptions. The 
problem of partial realization has been tackled by a number of 
alternate methods. For a definite treatment of these methods from 
the view point of abstract algebraic system theory, the interested 
person may refer to C143.0f great relevence to the present 
discussion , is due to KalmanC15,16] , which attempts to provide an 
algebraic structure to all formal power series, by describing them 
in terms of Jump points which constitute its invariants. Kalman 
shows that 

Ci^Every formal power series has a representation as an infinite 
continued fraction. 

CiiDA formal power series is rational if and only if its continued 
fraction representation is a terminating fraction. 

CiiiD All the invariants of the power series can be generated by 
Euclidean algorithm. 

Interestingly the same problem has been tackled in coding theory 
literature by Berlekarap-Massey algorithm[3,4] . The 

Berlekamp-Massey algorithm is a recursive design procedure for the 
design of a shortest linear feedback shift register CLFSR3 that 
generates a given sequence. One can find a complete panel lei of 
notion of jump and jump points in terms of LCnD vs n, where LCnD 
is the length of shortest LFSR that generates the sequence 
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th© partial sequence of the specified sequence 

a »a . »a ; n < N. In this connection we state some results. 

i 2 N 

Let a ,a , . . . . ,a be a specified sequence of lengtli N. TJi© 
problem is to design an LFSR of minimal length L as shown in 
Fig. 2.1 such that the first N outputs of LFSR starting from the 
initial state Ca^,a^» . . . . ,a^Dare exactly the specified sequence. 
An LFSR is completely char ec ter i zed by its length and the 
connection polynomial 

<yCx3 = 1 +0- x+ +cy,x*' ....C2. 2D 

JL l 

Let us denote it by CL.oCxDD.Let CLCnD, o'^’^^CxD be the 
shortest length LFSR that generates the partial sequence 

a^,a^ unknown parameters LCnD =1 , o'^'^^must satisfy the 

equation 


^ cX ^ » w ^^1 1 1 


1 

f 


< n> 

a., ■a.,,, 


m 



m 



<n> 




n-1 n 

. 

1 


If we denote the above coefficient matrix which has the 
Hankel form as ACn-1, ID', then we can prove the following results: 


1. LCnD=nx^ if and only if ACn/2,n/2D is nonsingular, 

2. LFSR C LC nD , xD D is unique if LCnD< n/2 or stated in! 
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another way if LCn3=m < n/2, then C LC nD , xD D = Cm,o*”'CxD3 

3. If n=21 ' and n=21Cl>l'D are two adjacent points where LCnD 
becomes n/2, then in the interval 21' < n < 21, LCnD jumps at only 
one point n = 1+1' and the amount of jump is equal to 1-1'. 

4. The LFSR CLCn!5, 3 has 2LCn3-n degrees of freedom in 

the sense that this many successive o'. 's may be chosen 

I 

arbitrarily. 


2.2 Berlekamp-MasseyAlgorithm: 

Referring to equation C2. ID 
if 

PCZD = p +p Z+ +p Z™ ... C2. 4D 

QCZD = q +q Z+ +q Z” 

O 1 T\ 

We normalize the coefficients such that q =1 . Now by forsnally 

o 

equating the terms in eq C2. 15 we get 

i=sn 

E Q. . =0 1 =n+l »n+2, . . . N . . . C2. 85 

. ^ V I - V 

t=o 

is=l-i 

Eq.a, . =p, for 1=1,2. m ...C2.65 

t=o 

The coefficients of QCZD are obtained using Berlekamp-Massey 
algorithm as explained below. 

The equation C2. SD represents an autoregressive filter, which 
can be implemented using shift register configuration as shown in 
Fig 2.1. To design this shift register we have to determine two 
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quantities. 

CiD The shift register length n, which is the degree of the 
feedback polynomial QCZD such that n should be minimum. 

Cii3 The feed back coefficients q ,q ,...q . 

The design procedure is recursive. For each r, starting with 
the iteration r=l, we will design a shift register for generating 
the sequence , a^ ,a^, . . . . a^. Shift register Cn^ ,Q* is a 

minimum length shift register. This conf iguaration need not be 
unique for a given length n^, and there may exist several such 
shift registers, all of the same length. At the start of the 
iteration r, we would have constructed a list of shift registers 

Cn .q‘^^CZ)D 
2 


Cn ,q‘'"^C2DD 
r 

The Berlekamp-Massey algorithm computes a new shortest length 

Cn .Q^’^'CZD!), that generates the sequence a a with the 

r 1 r 

initial conditions a ,...a . This is done by using the most 

1 n 

r 

recent shift register, modifying the length if necessary. 

At iteration r, compute the next output of the Cr-15th shift 
register . 


a 


j = l 


. . . C2. 7D 


Let A be the difference between the desired output a , 

r 


and the 
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actual output a^ of th® shift register. 


A 

r 



= a 

j=i ' 


equi val ent 1 y 


C2. SD 


j = l 



If A =0 then set 
r 

iteration is complete. 

If A O then n and 

r r 


. . . . C2. 9D 

Cn , Q^'‘*CZDD = Cn , Q.^ :> and the 
r r-l 

^ r* ^ 

Q CZ> are modified as follows. 


n =maxCn ,r~n 1 ....C2. 103 

r r - 1 r - 1 


q‘*‘’cZD= q“'“^’cZD+Az’'Q‘’"‘*’cZ5 C2.11D 

where, 

a=-a /a 

r m 
1 =r-m. 

m=:inaxi mum m. :such that m. <r and m O. 

0»e can easily prove that the new shift register Cn^ ,Q* ’C2D!5 

indeed generates the sequence a^ a^ with the initial 

conditions a ,a . For proof, interested persons may refer 

dL V 

to C43. 

The algorithm is described with the help of a flow chart as 
shown in Fig 2.2. 

From equation C2. lOD one can clearly observe the jump 
behaviour of n . The variation of the linear complexity by 



Knier 



Hah 


Fig 2.2. Berlekamp-Massey algorithm 
C Reproduced from [4] 3 
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advancing each iterat-ion is called linear complexity profile and 
is illustrated in Fig, 2.3. 

After obtaining the coefficients of QC ZD we substitute that 
in equationC2. 6D to get the coefficients of PC2D. Thus the partial 
realization is obtained. 

The Bor 1 ek amp-Massey procedure will have multiplications of 
the or dor OC n* D . 

2.3 Introduction of Error criterion: 

From the discussion in chapter 1 , we notice that most of the 
sequences will have higher complexity. Therefore, we expect not to 
achieve high compression ratios, using partial realizations, which 
give the exact sequence upon reconstruction. In this section we 
prove that that is indeed true ,and suggest an alternate method. 

2. 3* 1 Expected complexity of the partial realization procedure: 

Before evaluating the expected complexity of the partial 
realization scheme , let us first evaluate the distribution of the 
lengths of the minimal LFSK *s. The following discussion is from 
CIO] 

Lemma 1: Consider sequences of length n which take values from 
a finite field GFCpD. The distribution DCnjlD of the minimal LFSR 
's is given by 





1 


if 1=0 


DCn;lD = p if 1= 1, a ...Ca.l2D 

P p 1 f 1 = a+l n 

where, p = p-1 

cn = n/2 if n is even 

= Cn-lD/2 if n is odd 

Proof: The proof is by induction. 

Equation C2.12D is true for n=l . Suppose C2. 12D is also true 
for n=k: and consider vectors a ,a .a, ,a, . Here a, can 

1 2 k k+t Je+i 

take p values and p of these values cause the feed back equation 

to fail. For these cases 1 must be corrected. On the other hand 

satisfies the feedback equation for exactly one value and 

Ic 

then L. <a;> = L. CaD. It follows that p vectors retain the same 

k — 

length and the same set of initial conditions. The remaining p p 
vectors get a new set of initial conditions if there is a possible 
length change. Consider 3 cases, 
easel : 0< 1 £ a 

By induction there are p p vectors whose minimum length 
generators have length 1. Letting a arbitrary we have p p 


2i— 1— 

vectors a ,a >a, to consider. Of these p p remain 

4 2 k k+l 


of length 1; p®’"”*' p^experience a length change to k+1-1. 
case2; k+l -o«£ 1 < k+l 

By induction there are vectors a^,a^, ,a, of 

formula length 1; there is no vector of of formula length l=k+l. 
Adding component have p**^ '”***’ p vectors to consider. 

None of these experience a length change because 1> k+1-1. Summing 



20 


these two components we obtain p formulas of length 1. 

case3: ct< k-at =1 

This case occurs only when k is odd. We consider formulas of 
length l=a+l, the term missing in case2. By induction there are 
p of this length. Adding component obtain p p 

formulas to consider. All have length 1=«+1 because 2Cot+lD=k+l. 

Combining cases C1D,C25,C35 we see that equation C2. 12D holds 
with n replaced by k+l.By induction it holds for any n. a 
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expanding t,he data. However , since some error can be tolerated in 
the reconstructed sequence, we try to find a partial realization 
which, when expanded as a formal power series, gives a sequence, 
which is within some error limits of the original sequence. Thus 
we resort to a modified Berlekamp-Massey algorithm by introducing 
an error as described in [1,183. 

The Berlekamp-Massey algorithm gives the transfer function 
realization of a filter, which when excited by an impulse, 
regenerates the data exactly. It must be noted that at the Cr+lDth 
stage, if was the denominator polynomial CEq 2. ID to 

calculate a the predicted Cr+lDth value of the data, and it 

r 4- 1 

turns out to be equal to a ,i.e. if the error A =0, then 

r-t- i. r+l 

=q‘'’^C 2D and we forego the usual multiplication 
process, for the Cr+2Dth stage. This occurs rarely in a practical 
situation. We, however, allow for an error in the predicted 
val ues , such that , 

a -a < « ...C2. ISD 

r 4- 1 r + 1 

where s is the maximum allowable error. 

We now substitute a = a , A =0 and do not modify the 

r+l r + 1 r 

coefficients. If A >- s , we use the normal Berlekamp-Massey 

r 

procedure to obtain QCZD. The entire procedure is described with 
the help of a flow chart as shown in Fig 2.4 

The coefficients of PCZ) are then found using eq.2. 6 
Examples: 

CiD Error =0 


Modulo field =97 
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Desired sequence: 16,10,37,52,55 

Transfer function obtained using BM algorithm 

1 +84Z+9Z^ 

hczd = 

1 +68Z+752*+232® 

CiiD Error =15 Modulo field =97 

Desired sequence: 16,10,37,52,55, 

Transfer function obtained using BM algorithm 


1 

HCZD = 

1+77Z 

Generated sequence: 20,12,46,47,67,.... 
Error sequence: 4,2,9,6,12, ... . 




g 2. 


Modified Berlekamp-Massey Algorithm with error 
bound C Reproduced from CIS];) 








CHAPTER 3 

2-D PARTIAL REALIZATIONS 


In t-his chapter we study the representation of signals as 
impulse response sequences over a finite field of a 2-D linear 
shift invariant system. We present the basic structure and 
properties of the two dimensional systems, and describe Sakata’s 
extension of Berlekamp-Massey algorithm to two dimensions, for 
finding a partial realization to the given series. 

3.1 Structure and properties of two dimensional systems: 

In this section the basic structure and properties of two 
dimensional systems are briefly discussed. For a more detailed 
discussion interested fserson may refer to C83. 

The first contribution that discussed the problem of defining 
dynamical systems, was motivated by the necessity of investigating 
recursive structures for the two dimensional data, such as images 
and seismic signals. The idea of input -out put description of 
systems in two indeter mi nates as well as the design of analytic 
techniques based on the frequency response and two dimensional 
Z-transforms is well known. The new ideas that promoted further 
research in two dimensional systems for the concept of the state 
and its updating equations. 

From the beginning, very deep and substantial differences 
from the theory of dynamical systems in one variable have been 
evident. These are due to the mathematical tools, the concept of 
state itself, and to the structure of the state updating 
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equations. 

3 . 1.1 Two dimensional Input -Output maps: 

Lot T denot© th© cartesian product ZxZ, partially ordered by 
th© product of orderings, and introduce the notion of past and 
future of a point p := Cp^,p^3 in T as shown in Fig. 3.1. We call 
the past of p th© set 

= <Ci,j3; p> i, j » Ci,jD?«Cp,pD > ...C3. ID 

p 12 1 Z 

and th© future of th© point p, th© set 


3r = <Ci.jD; p^ < i . j> 

r 


, . . C3. 2D 



Fig 3.1 Past and future of a point which do not divide th© two 
dimensional plane into separation sets. 

Definition 3.1: A two dimensional system in its Input output 
form is defined as the set ,K,t^,3^»FD, where 

J" is th© time set defined above. 



%i s.nd ^ are t-he inputs and out/put^ spaces over a field K 
F is the input -output map. 

F is assumed to satisfy the following axioms. 

1. Linerity 

2. Two dimensional shift invariance. 

But such a concept of past and future in two dimensions is 

not sufficient for the description of the system because it 

doesn’t divide the two dimensional plane into two seperate sets. 

Therefore it is necessary that we introduce another notion of past 

and future so that the two dimensional plane can be divided into 

separate sets. This is necessary because we should be able to 

reconstruct every point to the future of Cm,nD, where Cm»n3 is the 

dimension of the system, using the past values upto the point 

Cm,nD. Kung et. al.C173, gave a definition for the past and future 

of a point. According to their definition the past of a point 

p=Cp ,p D is defined as 
12 

5<pD : = <Ci,jD I Ci+j> 5 Cp^+p^D> ...C3. 3D 

and the future of the point p=Cp^,p^D as 

.rCpD := <Ci,JD 1 Ci+jD > Cp^,p^D> ...C3. 4D 

The separation of the two dimensional plane using the above 
definition is illustrated in the Fig. 3.2. 



27 



Fig. 3.2 Past, and future of a point, which divide the two 
di mensi onaX pi ane i nto seper ati on sets 

Kung et. alCnJ , have proved that every formal power series in 
two variables can be expressed as rational approximation if and 
only if the associated Hankel matrix 


ci & 3> 

OO Oi lO 20 


a a 
Oi io 



is of finite rank. 

But even by this new concept of defining past and future, 
some of the problems will remain unresolved. Oie such problem is 
the inability to check the recurrence relation Ca relation by 
which we will be able to find the value of the sequence at a point 
using the past values till that pointD at some of the points even 
though they are the future points with respect to the dimension 
of the system. We illustrate this using an example. 

Consider a series Ca5. . i>0,j> 0;which can be reconstructed 
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using a linear recurrence relation represented by a polynomial in 
two variables which represents a linear recurrence of the 

form, 

^ ° Ci,j:>e^CC2,2:>D ...C3.5D 

Consider applying this relation at the point CS,0De;rCC2,2DD 
The required relation is a +a =0 

a , -2 5,0 

*a ,-2 defined. In order to avoid this problem we 

have two options. 

1. We can consider a. . to be O for i<0 or J<0. However it is 
not feasible since the system dimensions tend to increase. 

2. The approach we have followed is taken from the 
discussions of S:akataC20, 21 ,22,23] in the context of application 
to coding theory. Sakata has illustrated that a set of polynomials 
with degrees of each polynomial minimal, CThe actual definition of 
minimality comes later in this chapter 3 and a set of initial 
conditions based on the degrees of the polynomials will be 
sufficient to construct the remainig sequence. 

In this chapter we will study the approach used by Sakata. 
The problem that is attempted here is to find a set of linear 
recurrences of the form, 

E f a = O . . . C3. 63 

reT, *• 
f 

where s is the degree of the polynomial, q is a point in the two 

dimensional plane 2 s and a subset of the two dimensional plane 

over which f is nonzero, 
r 

Note: In the above equation r,q, and s are not scalars, but 
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two-tuples. If s := Cs^.SgD and t := Ct^.t^D.then s+t is defined 
as Cs^ +tj^ tSg+tg^ . 

The ordering > is defined as follows. 

s : = CSj^fSgD > t : = if and only if is not less 

than tj^ and s^is not less than tg. 

We have to f i nd a set of mi ni mal degr ee pol ynomi al and a set 
ot initial conditions which will determine the seguence uniguely. 

3.2 Definitions and Notations: 

Let K be any field. Over the 2-D lattice Z = a total order 5 

O O T 

is defined as follows. 

O := CO, 05 < Cl, 05 < CO, 15 < C2,05 

T T T 

and p £ g if and only if p<^ g or p=g 

One can number the points of the 2-D lattice with resp>ect 
to the total order defined above. For example n^CC0,055 =0, 

n^CC 1,05 5=1, n^C CO, 155=2 as shown in Fig. 3.3. 


Fig. 3.3. Total ordering on 

For a point p := Cp^,p^5 e Z^, the next point p+1 with 
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For an array A : — CAD Can array a with th® support, we 

o 

consider a linear recurrence relation 



0 



. . . C3. 9D 

where f e K 
r 

and 

Fj is 

a finite subset 

of 2^; for p:=Cp^,p^D, 

q:=Cq^,q^D e 

Z 

o 

, p+q 

:= Cp^+q^,p^4.q^D. 

With the above linear 


recur renco relation a bivariate polynomial may be associated 


f : = E f 2** « KCZ3 = KCz ,z ] ..,C3, lOD 

r, ’ 

q q 

where 2^ = is a power product of the independent 

variables z and z 
1 2 

and, 

Kt23 =KCz^,z^3 = Ring of bivariate polynomials over the field 
K 

The degree DegCfD of f is defined to be the maximum element 
of with respect to the total order provided that 

r ; = < q € 2 1 f € K 0> 

Definition 3.2: For a finite array A, f is said to be valid 
upto p if and only if Eqn C3.9D holds for any point q e 2^ 


Example: For a binary array shown in Fig. 3.S. , 
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10101010 

01100110 

lOlOlOlO 

00011110 

10000010 

01100110 

10000010 

OOOOOOOl 


Fig. 3,5. A E-D binary array 


Z 2 2 2 

the polynomial f=zz + z + zz + z + z 

12212ii 

with s = DegCfD = C2,2D is valid at any point aCi»jD >:s i.e. 


fCA] = a +a +a +a + a + a 

i,j i,j i-2,4 v,j-8 


.C3.11D 


3.3 2-0 Berlekanp-Massey algorithm: 

In this section the 2-D BerleJcamp-Massey algorithm is 
explained. Proofs of theorems mentioned in this section can be 
found in C223. 

Theorem 3.1: For a finite 2-D array A, if there is a 
polynomial g with degCgD = t, which is valid upto q, there is no 
polynomial f with degCfD £ t valid at every point r e and also 
at q. 

Definition 3. 3: A polynomial g of degCgD=t is said to havei 
spanCg!) : =q-t if and only if g is valid at every point r and 

not valid at point q. 
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Definition 3. 4: Excluded point set of A = Ca!) is defined by 

p 


ACAD = U A ...C3.12D 

e 

A^_^ : = I r:S q-t> if 3 a g with spanCgD = q-t 

: = $ otherwise 

. . . C3. 133 

Let C be the set of maximal points A^CAD 

Let r : = U r . . .C3.14D 

C c 

c€C 

We can associate a subset G of KCZ] corresponding to the set 

C of all maximal points ce ACAD^, i.e , every element c of C is 

spanCgD for some ge G. G is called the auxiliary polynomial set 

associated to the minimal polynomial set F e FFCAD . 

p 

Definition 3.5: For an array A VALPOLCAD is defined to be the 
set of all valid polynomials for the array A. 

VALPOLCAD is an ideal of polynontiial ring KCZ]. Thus if 
feVALPOLCAD, then gf eVALPCXLC AD for any geKCZ]. 

Our aim is to find a set of minimal polynomials if and only 
if all the following conditions are satisfied. 

1. F c VALPOLCAD 

2. S : = CdegCfD j fe F> is nondegener ate , where a finite 
subset S of Zq is said to be nondegenerate if and only if there 
are no distinct s and t e S and sS t . Therefore the set S will 
assume the form of a staircase as shown in Fig 3. ©. The region 

is denoted by ACFD also shown in Fig. 3.6. 



ACF3 





Fig. 3.6. The sets S and ACFD of a 2-D array CAD 


Remark: The set ACFD and hence S is unique for an array A, 

but the set F is not unique. Hence from here we denote ACFD by 

ACAD . 

P 

3.3.1 Description of the algorithm: 

The 2-D Berlekamp-Massey algorithm is an iterative procedure 

for finding a minimal polynomial set F e FFCAD and the set ACAD . 

P P 

Before going into the details the following theorem is stated 

without proof. Proof can be found in [223. 

Theorem 3.2: A CAD = ACFD. 

e p 

This implies that at point p we have an excluded point set, 

the maximal points of which are adjacent to the set S, the set of 

the degrees of valid polynomials at p. 

The importance of the theorem can be explained as follows. We 

check whether polynomials which were found valid till the point 

p-1 are valid at point p. If some of them turn out to be not valid 

the new auxiliary polynomial set and hence the excluded point set 

A CAD can be found. By the adjacency relation which is a 
e p 

consequence of the theorem 3.2, the new degree of the polynomial 


can be found. 
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Theorem 3.3: Let p<.j. q and g,f € KEZ] with degCg!)=t and 

degCf:)=s. If ge VALPOLCA3 , but gCa3 >*0 and feVALPOLCAD , but 

P M 

f C a3 0 i . e. > 

q 


E 

rti<sr^ 


J a 

m I 


9 


= O if 

= d 0 if n=p 


. . .C3. 16D 


E f' 

f 


a 

m m-hn-8 


= 0 if n^*’ 

8 

= d 0 if n=q 

q 


. . .C3. 16D 


then, 

h : = Z*— f € VALPOLCADq^^ 

wher o , 


. . .C3.175 


r :=Cr,rD is defined by 
1 2 

:= max<s^,t^+q^-p^>. := max<s^, t^+q^-p^> ...C3.18D 

Remark: t^+q^-p^, ^ negative. 

The polynomial h defined by the above equation may be written 


as h = hCf ,q,s,g,p»t3. 

An immediate consequence that follows from the above theorem 
is the following result. 

Theorem 3.4. Let s" e S be the elements of the set of 
degrees of f" . where f‘“e F ere the minimal polynomials for 

the array 

If p« ACADp, 


then there exists a polynomial h such that 
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h€ VALPOLC AD and degChD = 

Based on t.he above theorems we will explain the actual 
algorithm and its complexity. 

Define the following sets. 

F = I l<k< 1> c VALPOLCAD 

q 

S = <s“‘^=degCf‘’'^D | l<k<i> 

(Ic) 

G = Auxiliary polynomial set = <g | l<k<l-l> 

T = = degCg^’^^D \ l<k<l-l> 

Clc) 

PG = the set of points till which the polynomials g are 
valid = < p‘’*'^| 1 < k< 1-1> 

<lc> <tlc> 

DG = the errors at points p by using the polynomials g 
= l<k<l-l> 

The following provedure is an iterative procedure which gives 
the set of minimal polynomials valid till the desired point q. 

Stepl: n := CO,OD; F=<1>; G=^ ; DG = ^ ; PG = S=<CO.OD> 

Step2: begin 

stepCiD:find the polynomials which are not valid at 

the point n; 

stepCiiD : f ind the polynomials whose degrees are 
such that if n< s^''*+ACAD^, replace them by new h ’s 

stepCiiiD : For the remaining polynomials which 
satisfy stepCiD, but not stepCiiD; put them in a set the 

point n in PG , the errors in the degrees in S . 

Update the set G,PG>DG,T by 

NH 


G = G U F 
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PG = PG U PG 

NN 

DG = DG U IDG 

NN 

T = T U S 

NN 

st©pCvD:Find out the new from new P,G,and T. 

stepCviD: From the A set find the degrees of new minimail 

polynomial set by using the adjacency relation. 

CA CAD =ACAD -Z D 

• n+1 n+1 o • 

stepCviiD: from these new degrees update the polynomial 
set F by modifying the degrees and coefficients of polynomials in 
F by using the Berlek amp -Massey procedure described in theorem. 

stepCviiiD: Now all the updated polynomials are put in 

the set F. 

CThus all the sets of importance are updated. D 

St©p3: n := n+1; if n=q, then stop; else go to step2: 

Sakata has proved that this algorithm has a computational 

z 

complexity of the order OCn D. 

Now the sets F , SC or AC FDD » with the initial conditions A 
with the support ACFD uniquely determine the sequence A upto the 
point q. A sample computation of the algorithm for the sequence 
shown in Fig 3.7, is shown in table3. 1. 

1 0 0 0 .... 

0 0 0 . . . . 

11.... 

1 . . . . 


Fig. 3.7. A binary 2-D array 
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Table 3.1 Sample computation of 2-D BM algorithm for the array 

shown in Fig. 3.7 


n 

F 

G 

p 

CO.OD 

<Z ,Z > 

1 2 

<1> 

<CO,OD> 

CO. ID 

<z .z > 

1 2 

<1> 

<CO,OD> 

Cl.OD 

<z .z > 

1 2 

<1> 

<CO,OD> 

C2,0D 

<zr*+i ,z > 

i 2 

<Z^,1D 

<C2,0D ,CO,OD> 

Cl 

CZ^+l ,z > 

A 2 


<C2,OD ,CO,OD> 

CO, 2D 

<Z^+l ,z > 

1 2 

<Z^,1> 

<C2,OD ,CO,OD> 

C3,0D 

<2f+Z +1 ,Z > 

A A 2 

<Z^,1> 

<C2,0D , CO, ODD 

C2,1D 

<Z*+Z*+2 ,Z Z +Z ,Z*> 

A A A A 2 A 2 

<Z*+Z +1 ,Z > 

A A 2 

<C2,1D ,C2,1D> 



CHAPTER 4. 

APPLICATIONS OF PARTIAL REALIZATIONS TO DATA COMPRESSION 


The objective of the present cha.pter is to study whether 
pa.rtia.1 re&liz&tion schemes can effectively be used to compress 
the data. We have applied this coding scheme on the image data. 

4.1 Applications of 1-D partial realizations to image coi^pression: 

The coding scheme used here is the representation of signals 
over a finite field, which represent some realistic data such as, 
images etc. , using 1~D partial realizations with error criterion. 
The sequence is represented by the coefficients of the polynomials 
PCZ>,QC2D. We have applied this coding scheme on image data. The 
image we have taken is a 64x64 image with 32 gray levels. The 
field in which the coefficients were found is GFC31D. Coding is 
done individually on blocks of data of length 8. These blocks were 
scanned horizontally. The conqpression ratios obtained for various 
error values are shown in table 4.1 

The reconstructed images are shown in figure 4.1. If we look 
at the table 4,1 we observe that the coa^ression ratios obtained 
are very small. It is also possible that this sort of coding may 
lead to data expansion. Since the expected complexity analysis is 
very difficult with the introduction of the error criterion we 
have run some simulations on the computer , We have taken sequences 
of length 8 over a field GFC53 and evaluated the distribution of 
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Tabl© 4.1 Compressions obtained for various error values using BM 

^iQorithm with error bound. 


Error value 


a 

4 

5 
7 


Compression 

% 


6 . 6 
8 . 8 
11. 7 
11.9 


the number of sequence for each length of the LFSR. We have chosen 
an error of 1. The results are shown in table 4.2. 


Table 4.2 Distribution of LFSR lengths using BM algorithm with 

error bound 

Length of the LFSR Number of sequences 


0 

1 
2 

3 

4 

5 

6 

7 

8 


256 

5576 

13542 

54072 

188080 

100513 

22258 

5244 

384 


Now if we compute the expected complexity E of the code, then 




C aD Or i gi nal i mage 


CbD Compressed image 
with err or =2 


CcD Compressed image 
wi t h er r or =4 


CdD Compressed image 
with err or =5 



CeD Compressed image 
with err or =7 




Fig. 4.1 Compressed images 


using BM algorithm with error criterion 
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Isn 

E = C 2 21DCn;lD D /p” 
uo 


. . .C4.13 


where DCn;13 is the number of sequences having a code length 
1, n is the length of the sequence » and p is the cardinality of 
the alphabet over which the sequences are defined. 

Substituting in equation 4.1 the values of DCnjlD we get the 
value of E as E =8.3 > 8. Thus we see that we cannot achieve 
compression even by introducing the concept of error criterion. 
Therefore, the following encoding schenra is chosen. 

If the code length C number of coefficients of both numerator 
and denominator of the rational approximation, which is equal to 
twice the length of the LFSRD is greater than, the sequence is 
represented as it is. We encode only when the code length is less 
than the length of the sequence. Using this approach we got better 
results, as shown in table 4.3. 


Table 4.3. Compressions obtained for various error values using 
modified representation of sequences using partial realizations 

with error criterion 

Error value Compression 

Vo 


2 

4 

5 


22 

24 

26 


7 


28 



Ca.D Original image 


Cb5 Compressed image 
with error =2 



CcD Compressed image 
with err or =4 



CdD Compressed image 
with err or =5 



C eD Compressed image 
with err or =7 


Fig. 


4.2 Compressed images using modified representation using 
F>artial realizations with error criterion. 
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4.2: Application of 2-D Partial realization to compression: 

Application of 2-D partial realization schemes to compression 
have yielded very bad results. It was found that this scheme 
without introducing the error criterion will in general expand the 
data. Where as, this method is very useful for the decoding of 
multidimensional codes, they are ingeneral found to be very bad 
choice for data compression. This may be attributed to the use of 
a set of polynomials rather than a single polynomial. But as 
discussed in chapter 3 a single polynomial will not suffice to 
describe the entire sequence. Introduction of error criterion is 
also very difficult, because, while we aim to reduce the degree of 
one polynomial the degree of other polynomial may increase. Hence 
it can be concluded that 2-D partial realization is not a very 
good choice for partial realization. 

4.3. Drawbacks of using partial realization schemes for data 
compr essi on: 

In the previous sections the application of partial 
realization schemes to data compression is described- From here 
onwards the discussion will be confined to 1-d case. 

From the application of these algorithms to the image data 
the following observations are made. 

CiD Higher compression ratios cannot be achieved even for 
high values of the errors. 
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CiiD The compresion ratio may not increase by increasing the 
value of the error. 

In this section we prove that these representations are not 
optimal in the sense, that they are not. minimum realizations such 
that the reconstructed signal is with in the error limits. 

Notice that the linear complexity profile as described in 
chapter 2 exhibits a jump behaviour while using the 
Berlekamp-Massey algorithm. Suppose that at the i — th stage the 
number of shift registers needed for the transfer function 
realization are 1. Assume that at stage r, 1 to be less than 
Cr+lD/2. At the Cr+13th stage if the predicted value of sequence 
aCr+lD is not equal to aCr+lD we find a new connection polynomial 
the degree of which jumixs to Cr+l-i;> which is greater than 1- This 
Jump behaviour is illustrated in CfigD 

We now proceed to prove that introduction of an error 
criterion as described in chapters 2 and 3 need not give an 
improvement Infact it is possible that it may even give worse 
results. 

Suppose that we are finding the connection polynomial QfCZ> 
with an error e. At the r-th stage let the number of shift 
registers needed be 1^. Assume that 

1^*‘>< l'*** < Cr+lD/3 ...C4. 2> 

e o 

where, 

= number of shift registers needed to reconstruct the 

o 

exact sequence till r— th stage. 

Let the predicted value a^^^ 


at the Cr+13-th stage differ 
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from a by more than € , i . e , 

r+l 


a - a 

r+i 


. . . C4. 3D 


In this case the degree of the connection poljmomial at the 
Cr+lD-th stage with error is 


, <r+l> 

1 = r +1 


. . . C4. 4D 


and that without error is. 


, <r+i> 

1 = r+l 

o 


. .C4. 5D 


From equations C4.2D,C4. 4D and C4. 5D it is clear that 

, { r+lD . , < r+iD _ . x . . ^ 

>1^ . It means, that, for some sequences it may be 

possible that, the degree of connection pol>nomial with error 

criterion is greater than that, without the error. However, since 

such cases occur rarely, we expect on the average, that partial 


realizations which allow for some errors do achieve better 


compression than, the same methods which do not tolerate any 
error. This also explain why the compression ratios do not 
substantially increase with errors. 

Since the Berlekamp-Massey algorithm does not give optimum 
results with an error criterion we search for an alternate 
formulation of the problem. Instead of updating the coefficients 
sequentially such that the predicted values are within the error 
bound, we pose the following question. Which one of the sequences 
which are with in the error bound has least linear complexity. 
That is, we want to find a transfer function realization of the 



form 


PC ZD 
QCZD 


. . .C4. 6D 
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such that, 

/s 

I a. -a, I < s 

and the degree of QCZD minimum. 

As the following theorem states the computation required to 
realize such a system must have an exponential con^lexity, which 
cannot be carried out by a nondeter mini Stic polynomial time 
al gorithm. 

Theorem 4.1: Given a sequence a ,a over a field 

12 n 

GFCq5 and an error limit e such that £: << q the problem of finding 
a minimal linear recurrence equation with least degreee such that 

yS /V ys 

the reconstructed sequence within the error 

limi ti . e. , 

/N 

I a.-a. I 5 45 V ie <1 ,2, . . . . ,n> 
i s NP-hard 

Proof : In the worst case there are C2«+l!)” sequences which are 
within the error limit. In order to find the sequence that has got 
least linear complexity, we have to enumerate each of these 
sequences by Berlekamp-Massey algorithm Cthe B-M algorithm is 
optimal in number of computations to find the linear recurrence 
relation^ and find the one with minimum linear complexity among 
these. Since size of the input itself is exponential in n, this 
problem is NP-hard. Therefore there is no other way to tackle 
this problem except by using a brute force method. 

Using brute force approach we have coded the same image 
referred earlier. This approach gave a very high improvement over 
the previous two cases. Where as previously we could get a 
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compression ratio of 2S?^ for an error 3 now we could get, 
compression ratios upto 805i. The compression ratios obtained for 
various values of error are shown in table 4.4. The only problem 
with this approach is that for large values of error the 
computation is infeasible. 


Table 4.4 Compressions obtained using brute force method. 


Error value 


1 

2 

3 


Compr ession 

38 

47 

S8 



C aD Or i gi nal i mage 



CcD Compressed image 
with err or =2 


Fig. 4.3 Compressed images \ 




Cb5 Compressed image 
with error=l 



CdD Compressed image 
wi t h er r or =3 


ing brute force method 



CHAPTER 5 


CONCLUSION 

In the absence of knowledge of the statistics of the signal , 
it was generally observed that data compression is difficult to 
achieve. This fact was illustrated by using Chaitin's example. 
This difficulty arises because » the expected complexity of the 
sequence cannot be less than the entropy. However by using a 
fidelity criterion which for the objective at hand can be 
tolerated by a machine or human, it was conjectured that some 
compression will be achieved. 

The signals that were considered in this thesis were 
sequences over a finite field GFCpD where p is a prime. The 
signals have been represented using partial realizations. The 
error criterion employed here was to place an error bound on the 
sequence, so that each value of the sequence estimated by this 
representation will be within the error bound of the original 
sequence. We have considered two cases. 

1 . One di mensi onal par t i al r eal i zati ons . 

2. Two dimensional partial realizations. 

In first case the modified Berlekamp-Massey algorithm with i 
error proposed by Reed was implemented. Application of this ; 
representation to image data showed that very little compression , 
ratios would be obtained. By running some simulations it was shown | 
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that, the data may actually expand using this representation even 
with an error criterion. A slight modification was suggested to 
this representation, that, when the length of the representations 
C number of coefficients needed for the representation^ exceeds the 
length of the sequence, the sequence is represented as it is. It 
was illustrated that the representation using the error criterion 
may, sometimes give worse results than the representation without 
error, though, for very limited cases. An alternate formulation of 
the problem was chosen which seeks a representation with minimum 
number of coefficients such that the reconstructed sequence is 
within the error bound. It was proved that this problem is NP-hard 
and is computationally intractable, for large values of the 
sequence lengths and error bounds. However, the brute force method 
was tried on an image data for small values of the error. 
Compression ratios upto 60?^ were obtained, but with a very high 
computational cost. 

2-D partial realizations using Sakata’s extension of 
Berlekamp-Massey algorithm to two dimensions was implemented. The 
application of this representation to image compression yielded 
unpromising results. This may be attributed to over complex-view 
of the system under observation. Introduction of error bound is 
very difficult, because when one seeks to reduce the degree of one 
polynomial, the degree of other polynomial may increase. However, 
this algorithm has got impxsrtant applications in the decoding of 
two dimensional codes, e. g. ,2— D fire codes. Since the feedback 
polynomial set forms a Groebner basis, this algorithm may also be 

.It ,..t\ 
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used for constructing a basis for two dimensional polynomial 
ideals. 
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APPENDIX A 

CORRECT PROOF FOR THE EXPECTED NUMBER OF STATES FOR 

GAINES MODEL 


In 

section 1 

■ 2.2 it was 

pointed 

out 

that the 

proof 

by 

Gaines, 

given for 

deriving a 

bound on 

the 

expected 

number 

of 


states was wrong. The correct proof is given below. From equation 
(1.4) the average number of models for S-1 states is given by 

S-1 

M(S-l) « E 3 ...(A.l) 

S, -1 R=1 


Therfore the average number of S state models is obtained by 
simply replacing the S-1 in the above expression by S, and after 
some algebraic simplification , we get, 


M(S) = ((S-1) )/2® 

S S2 


. . . (.A.2) 


If /jg is the average number of states, then 
A/q 2® > Mg2 “ = M(S) 

From (A.2) and (A. 3) we get 

Mg > S-2 


...(A. 3) ; 



F-rom (1,2) and (A.4), 


p > N-:ioq,.,N -2 


, • ■ ( A ■ 5 ) 


The ratio of the expected number of states to the number 
of observations is given by 

R = > (N~1oq,^N -2)/N ...(A. 6) 

N ^ 


and N-+OC. This is the result obtained by Gaines though by 

a falty proof. 



.’ll j 


t .i: !i" 1; !{.■ !f: 1 1 )!•: $ if: if: t S 1 1 1 1 1 1 1 1 $ t if: t ij: V $ t -’J- f if >;■: if: il; if: t ii: sli Jf; t if. ifi t 

I':) ro Cl ram ci-lvos a set of m.ln.iitial po.lvn<;:iiii,i. a.ls ■S< 


wli.ic h 'vii't.isty t.!"'ie two cl.i.men<s.tona 1 1 .inear reourrence 
r 0 .i.at.iDn for a given two dimensional array. This * 

usf.'s Bakata"s eMtension of Berle?kamp Massey rilgorithm 5i< 
for the two dimensinal case. !f: 

if; t if: 1 1 if, ;f; )i: 1 1 i|; if 1 1|.: H; if if; H; 1 1 $: if: if; if: t « }!•: t if: si: !l; $ 1 1 sf: t * t if; t :!•: >1; Jf :f: ;!; ;f: if: U; if: if; ;!< if; if; 


11 inc lude "inf i 1 " 
main ( ) 

f 

/t DECLARATIONS t/ 


/ if if if if if if if if if if if $ if $ if if .K if: s t «■: if if T -f: t $ if 1 1; $ if: t if if t if if if 1 1 :f if: 1 1 if if * :f if / 

int ifh, ii<f,, ifg, iffl >fgl!i 

/% h,f|,g are the pointers for the netjpc:>lynomi al , 

the current polynomial sot and the auiriliary 
polynomial set respectively 
/% f.l, gi provide the working space %/ 
struct >:,.....y„z ■[ 

int MS 

i n t y 5 

int if:<r. 5 

.} 5 

struct iL....y „..2 slTMPOLl!, tfMMPGLJ, c [MPOI...".! , r, p, q; 

/if s - dogree set of tlie currsint polynomial set 

t =" degree sot. of auMiliary pcilynomial set 
c mai-! point set of c-'kc luded point set i!</ 


s t rue t M !•:> .1 1: MPOI... ] , 1: 1 f. NPOL ] , c i H Ni hDL ] ; 


int 

■i *1 .j 3 k ii Is 


i n t 

dq, dg 5 


int 

validity! ) checksdeltaC ) s 

int 

nej!t(), order 

structure ! ) 5 

int 

order_array( ) , 

ad.;j acency ! ) 5 

int 

berlekampC ) ; 


int 

auxiliary ! ) 5 


int 

copy_pol ! ) , cDpy_.,struct ! ) ; 


/ if if If if if if if if # » * if if * if if if if * H: if if: if if if « )}• if * * if * # if * if ^ if !v: ’i' '1^ * * ’■I' ^ * ’I' ■ 


f 

= cal loc !si 2 :eof 

! int ) if:ARSZ 

if 

ARSZ if NPOL 

<3 

» cal loc ! si 2 :eof 

( int ) JKAF^SZ 

t 

ARBZ if NPOL. 

i-i 

=5 cal loc ! sizeof 

! int ) if Ar;Bl;; 

t 

ARBZ' > ; 

f.i 

= c a 1 1 oc ! s i z e 0 f 

! in t ) ifii iR'oZ 


Af-itoZ if: MPOi... 

gl 

= Cell loc ! sizeof 

! int ) $ ARSZ 

t 

ARSZ if i-iPOL.. 


iff = .1.5 



for ! i =- 

0;; i ’= 

:; 10 ; i '- + ) { 



““ f -i- ARBZ ARSZ JK 
t|.;iUz ^ g ARSZ t ARSZ t i? 
c[:i;i-z === q -t- ARSZ t ARSZ i ;; 



p - s 

P - Y 0 ;; 

ne^t ( p) I! 

I ““ 0 ; 

poinl cDurrl. - 1 r, 

if ( paiivl.,, coun t < P_^CGUMr) i 

c cl n ■ j C ' ’ “ H (;:! ' ' ^ ^ a I p o i n t _c o u r *} t . x ] p o i n t _c □ u n t _y II ) 5 
int iN 05 

irit iV - O5 

ror (j =-• 0; j 0= 15 { 

dq r j 3 ~ valid! ty ( i?<s [ j 3 a n ' p ) 5 

If (dq[:j 3 0) C 

sVL:iV 3 -x s[:j 3 -Mii 

BVi:;iV 3 «y s[.j 3 -yii 

1 V“h 4 “ 5 

J' €S 1 S0 ■(. 

if ( ( c hec k - c: he^c k sde 1 1 a ( &s [ j 3 ? Q ) ) -=== 0 

sM[iM 3 « =•“ sC j 3 - >< |} 
sNCiN 3 . y = si;j 3 .y!i 
tNCiN 3 .x tN[;iN 3-)<5 
tNCiN 3 «y =•*= tNCiN3-y5 
iM 

} else { 

int tmpl, tmp2|i 

int tmp “s Oil 

for (tmpl » O5 tmpi < sCj3«x; tm 


J 3 M y s 

) i 

RSZiK At:BZ 

3 ARSZ 4* si;j3.y3 

..1 3 - ; 

J 3 « y !i 

p « )< - s 1 : j 3 - M s 
p - y s i; j 3 « y ? 


' for (tmp2' =• On tmp2 < sC 

/ 

gl C ( l'+-tmp-l) He A. 

f C ( j -i ) )KARBZ)KARBZ 4* s[ j 3 . x HCARSZ-^sC j 3 y 3 S 

tl C l4-tmp3 - X === 'sl.“ 

tiCl 4 “tmp 3 -y == sC 

clCl-Htmp3-x 

clCl 4 '’tmp 3 *y 

tmp 4 -h |5 

n. 

J 

"J. 

J 


now! - order^structureCcl J 1 4 - tmp)|i 
ad j acency (sip c i , new 1 ) 5 
Q r (" 1 0 r a r r a y ( 1 1 ) 5 



i.l ) 


roi' (i - 0; i < newl ; i+-i-) ■[ 

for (j = 0; .j < iVs j-i-i-) { 

if — bVCJ].?-!) 

' for (ii = 05 il < s|Ii].K 


i, ;i. .1 


;i l + 'i- ) 


‘for (jl « O 5 jl 


* ( B1 C i "J . 

■ 1 - 


ARSZ %'■ .i 1 + j 1 ) 


- 1 - AR5Z t il 4- jl) 
1 


bN 1 : j ] , bN C .. i 3 . ;•! j s C j 3 , t C j 3 -> 2 


for (j = 0 ; j 
for (j 


p, q, dqCj3); 

.} 

•V 

J 


iN; j-i-+) { 

= O 5 j < ilM; j4-+) i 

if (si Hi 3 . 5-: === BN[;j3.x) 

berlekampC h. 


polynomials a r e 

neocied to nivo the new 
procedure^/ 


/* otherwise if none of the auxiliary 


polynomial then use the auxiliary 
auxiliary {h, siCi3ii sCi3); 


copy fl to f; si to s; gl to g 5 tl to t;,’cl t 

c; c:iha\t to chat; t/ 

copy,„pol ( f 1 , f); 

- c a py ._.s t r uc t ( s 1 , B ) ; 

copy_.pol(gl , g); 
copy__struct ( tl , t); 

CDpy_struct ( cl J, c); 
copy_,struct ( clhat, chat); 

1 = newl ; 

nextv p) ; 


for (i == 0; i < 1; i+"i-) { 

for ( ..i -• O ! .i < s C i 3 . x ; j 4-+ ) 

for (k =0; k < s[i3“>y; k++) 
printf("%d", flIj3Ck3); 

printf ( " \n " ) ; 

.1 

free(f); 
f ree (q ) ; 
free(h )5 
free(f.l); 
f I' ee Cql ) ; 


X 

J 



^I'iric; .i.ude " inf i 1 " 

i ! i ' bo r 1 cv? k ai!i p ( |-i r , f , b , g , t , p , q , d q ) 

irrt ^i(h, i!<f, :!£q; 

;:il' I" u(.: i. x_ _y z 1 

irvi: >; 5 

ini v; 

ini: Ss; 

i if, ifcn , -it, f.p, iKq; 
a.ri L dq ; 


nqwpoly ; deg ( h ) =r |i i;/ 

/^f‘-poly to be modified deg(f)--B ord(f)=q-5*/ 
/i^ci=mai-! eKpoly deq(g) = t ord(g)=p-t*/ 

/!}tdq™dei 

struct !•; y^z $1; 

.1. 1 T Lt i , .3 If k |i 

int )f:temp; 

int copyi ) , : 2 _mult( ) , sc__mult( ) , poly_sub( ) i; 
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mal loc ( sizeof 

(int)5icARSZ * ARSZ) 

5 

( r) - > 
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( t ) “> >! + ( q ) -> X + 

( p)->x ) 5 
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y 
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(t)->y + (q)->y + 

(p)->y ) 5 

( !)•"> 

X 

= (r)->x - (s) 

->x ; 


(1)”> 

■y 

~ (r)“->y (s) 

~>y j 



copy(h, f, s )5 
z,.,.,mal t ( h , s, I); 
copy ( t.(s(np , g , t ) ; 


SC, nu.Ai t ( temp , 

t, dq )5 



{l)->x - (r)-~- 

> X “ ( q ) - > 'A + 

(p)“ 

>x - (t)">x; 

(U->y (r)- 

>y •“ (q)“>y + 

Cp)- 

>y - (t)->y3 

z 1 1 ( 3 

t , 1) ; 



poly sub ( h ^ r 

, temp); 




f re's C temp ) n 
return ( d ) ;; 
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i !'■ .1 n c: 1 u d e " i, n f i 3. ' ' 

int au>;.i 1 iarv ( h , r ,, 

in L !!cf j ijcg; 

stri.ict !■( y i: f 

i n t !•; 5 

int y 5 
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i i f ; S j 


/ 1 i -I ••== 1 1 e p o 1 y !| d e g ( 3i ) = r 5 
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/!((dq==del taiK/ 


struct Jl; 

i n 't i , j k ; 

int !i;temp5 

int. copy(), z.jnultd)., sc„rnuitC)ii poly_sub()5 

temp ~ rna 1 1 oc ( si seof (int) * ARSZ # ARSZ ) 5 
( 1 ) -••• > =" ( r ) -• > K - ( S ) -• > X !! 

( 3 . ) •■•->>' (r)->y “ (B)->y5 


c o |.j y i. i 1 3 * 5 ) ji 

z...(fiu3. t ( B, r); 
•free ( temp ) ; 
return ( 0 ) ; 



tl include "infil" 

int copy(h5 Lernpj, p) 

struct z i 

i. r-i L. :•! ; *■ , 

int y; 

int i|iz; 

:!■ !}: p 5 

int thf 

/■ 

1. 

int i, j; 

for (i = O5 i <= p-->;-!.5 i++) 

for (j O 5 j <= p->y5 J"'"'") 

S:(h + (ARSZ !K i) + j) = !K ( tempC ARSZ^i + j ] ) 


return ( 0 ) ; 
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f i. .1. " 
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1 1 1 1:. t It , 

sKf ; 

Btr'iAt;; L m z 

r 

}, 

j. n t 

/\ »') 

ii'it 

y *” 

in t 
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,r 
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i n t 

i 1 J 5 

in t 

(nod ( ) ; 

for 

( i = 0 5 i 


for ’(j 
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= r— >x ; i++) 

= 0; .1 <== r->y; j ++ ) 
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ARSZ Sic 
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de "inf.il" 
z muilKri, r 
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i. n t 

w 
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i n L 

V' il 
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)' n'- r* r| r s 5 

.1 I'l 1 : ' 1 ^ 1 1 ;; 

r 

int 

i , j 5 


in t 

!{:temp 5 


sir uc L, 

:•! y 

h-*‘ 5 

in t 

copy ( ) 

5 

temp == 

ma 1 1 oc ( ‘ 

■aizeof (int)>i£ARSZ 

for (i 

0 s i 



for (j 
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for ( i 

” ■* r.i;- M 5 
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for (j 

“• O 5 .j < S">y!i j 
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for ( .i. 
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'1 ui ■ ( j 

"" r:.— >y; i 
t i temp ARSZ 
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copy ( I'l 
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